Fast Filtered Sampling

Please cite this article in press as: J. Mai et al. Traffic sampled from the network backbone using uniform packet sampling is commonly utilized to detect heavy hitters, estimate flow level statistics, as well as identify anomalies like DDoS attacks and worm scans. Previous work has shown however that this technique introduces flow bias and truncation which yields inaccurate flow statistics and ‘‘drowns out” information from small flows, leading to large false positives in anomaly detection. In this paper, we present a new sampling design: Fast Filtered Sampling (FFS), which is comprised of an independent low-complexity filter, concatenated with any sampling scheme at choice. FFS ensures the integrity of small flows for anomaly detection, while still providing acceptable identification of heavy hitters. This is achieved through a filter design which suppresses packets from flows as a function of their size, ‘‘boosting” small flows relative to medium and large flows. FFS design requires only one update operation per packet, has two simple control parameters and can work in conjunction with existing sampling mechanisms without any additional changes. Therefore, it accomplishes a lightweight online implementation of the ‘‘flow-size dependent” sampling method. Through extensive evaluation on traffic traces, we show the efficacy of FFS for applications such as portscan detection and traffic estimation. 2010 Elsevier B.V. All rights reserved.